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(57) ABSTRACT 

Computer programs and computer-implemented methods 
are disclosed for extracting and analyzing combinatorial 
chemical data from images. The invention receives a stream 
of data, identifies a plurality of regions of interest and 
determines a series of values for one or more regions of 
interest. The library includes a plurality of members. The 
data stream includes a scries of images of the library. Each 
region of interest corresponds to a location in each of the 
images and to a location in the library. The series of values 
for each region includes a value for each of the images. 
Particular embodiments have the following features. The 
invention calculates a figure of merit for a library member 
from each series of values. The stream of data comprises a 
series of images generated at a frequency of greater than 
about 1, 6, 12 or 30 frames per second. The scries of images 
is captured from a beginning of a combinatorial experiment 
to an end of a commercial experiment. The regions of 
interest arc identified alter the series of images has been 
captured. The images arc infrared images. The figure of 
merit is an einitlancc change, a phase transition point or a 
thermoelectric figure oi merit for a library member. Regions 
of interest are identified by a user and have a user-defined 
shape. The scries of values for each region is determined 
based on a user-defined computation. A graphical represen- 
tation of the figures of merit is displayed. 

42 Claims, 16 Drawing Sheets 

(1 of 16 Drawing Shcct(s) Filed in Color) 
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CROSS REFERENCE TO RELATED 
APPLICATIONS 

This is a continuation of and claims priority to Interna- 
tional Application PCT/US99/07358, with an international 
filing date of Apr. f, 1999, now abandoned, which is a 
continuation-in-part of commonly assigned, co-pending 
U.S. patent application Ser. No. 09/227,558, filed Jan. 8, 
1999, which is a continuation-in-part of U.S. application Ser. 
No. 08/898,715, tiled Jul. 22, 1997, and U.S. Provisional 
Application Nos. 60/050,949, filed Jun. 13, 1997; 60/028, 
106, filed Oct. 9, 1996; 60/029,255, filed Oct. 25, 1996; 
60/035,366, filed Jan. 10, 1997; 60/048,987, filed Jun. 9, 
1997; 00 1)28,105. filed Ocl. 9. I 996; and 00 035,202, filed 
Jan. 10, 1997. Each of the foregoing applications is incor- 
porated herein bv reference in its entirety and is the basis of 
a claim for priority under 35 U.S.C.§if9 or 120. 

A portion of the disclosure of this patent document 
contains material which is subject to copyright protection. 
The copyright owner has no objection to the facsimile 
reproduction by anyone of the patent document or the patent 
disclosure, as it appears in the Patent and Trademark Office 
patent file or records, but otherwise reserves all copyright 
rights whatsoever. 

BACKGROUND OF THE INVENTION 

The present invention relates to methods and computer 
programs for rapidly screening and characterizing materials 
by the analysis of data acquired from images. 

In combinatorial chemistry, a large number of candidate 
materials are created from a relatively small set of precur- 
sors and subsequently evaluated for suitability for a parlicu- 

istry permits scientists to systematically explore the 
influence of structural variations in candidates by dramati- 
cally accelerating the rates at which they are created and 
evaluated. Compared lo traditional discov ery methods, com- 
binatorial methods sharply reduce the costs associated with 
preparing and screening each candidate. 
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In general, in one aspect, the invention features a com- 
puter program for evaluating a combinatorial library includ- 
ing a plurality of members. The program includes instruc- 
tions to receive a stream of data including a series of images 
5 of the combinatorial library; to identify a plurality of regions 
of interest, each region corresponding to a location in each 
of the series of images and to a location in the combinatorial 
library; to determine a series of values for one or more 
regions of interest, the series of values for each of the one 
10 or more regions including a value for eaeli of the images, 
and to calculate from each series of values for the one or 
more regions a figure of merit for the library member at the 
corresponding library location. 

Implementations of the invention can include one or more 
is of the following advantageous features. Each region of 
interest corresponds to a plurality of pixels in each image 
and the scries of values for a region of interest comprises an 
average value for each of the corresponding plurality of 
pixels in each of the images. The stream of data comprises 
j, a series of images generated at a frequency of greater than 
about 1 frame per second. The stream of data comprises a 
series of images generated at a frequency of greater than 
about 6 frames per second. 1'tie stream of data comprises a 
series of images generated at a frequency of greater than 
;s about 12 frames per second. The stream of data comprises a 
series of images generated at a frequency of greater than 
about 20 frames per second. The program further includes 
instructions to display a graphical representation of the 
figures of merit. The graphical representation includes a 
> histogram. I he series of images is captured from a begin- 
ning of a combinatorial experiment to an end of the com- 
binatorial experiment and the plurality of regions of interest 
may be identified after the series of images have been 
captured or after one or more images in the series of images 
35 has been captured. The scries of images includes a scries of 
infrared images. The figure of merit comprises an emitlance 
change for a library member. The figure of merit comprises 
a phase transition point for a library member. The figure of 
tompriscs a thermoelectric figure of merit for a library 



Combinatorial chemistry has revolutionized the process 40 member. The instruct ions to identify a plurality of regions of 



of drug discovery. One can view drug 
two-step process: acquiring candidate compounds through 
laboratory synthesis or through natural products collection, 
followed by evaluation or screening for efficacy. Pharma- 
ceutical researchers have long used high-throughput screen- 
ing (II IS) protocols to rapidly evaluate the therapeutic value 
of natural products and libraries of compounds synthesized 
and cataloged over many y ears. However, compared lo HTS 
protocols, chemical synthesis lias historically been a slow, 
arduous process. With the advent ol combinatorial methods, 
scientists can now create large libraries of organic molecules 
at a pace on par with HTS protocols. 

Recently, combinatorial approaches have been used for 
discovery programs unrelated to drugs. For example, some 
researchers have recognized that combinatorial strategies 
also offer promise for the discovery of inorganic compounds 55 
such as high-temperature superconductors, magneto resis- 
tive materials, luminescent materials, and catalytic materi- 
als. See, for example, U.S. Pat. No. 5,776,359 and 
co-pending U.S. patent application No. 08 898,715 "Com- 
binatorial Synthesis and Analysis of Organometallic Com- 60 
pounds and Catalysts" (published as WO 98/03251), which 
are both incorporated herein by reference. 

SUMMARY OF THE INVENTION 

The invention provides computer programs and 65 
computer-implemented methods for extracting and analyz- 
ing combinatorial chemical data from images. 



comprise instructions to receive a first user input 
identifying one or more regions of interest, each region 
having a user-defined shape. The program further includes 
instructions to receive a second user input defining a com- 
putation foi determining a sctics of values for each of the 
one or more regions of interest. The instructions to deter- 
mine a scries of values for each of the one or more regions 
of interest include instructions to compute each value in 
each series in accordance with the second user input. The 
computation is an average for a group of pixels correspond- 
ing to a region of interest. 

In general, in another aspect, the invention features a 
computer program on a computer-readable medium t'or 
evaluating a combinatorial library including a plurality of 
members. The program includes instructions to receive a 
stream of data including a series of images of the combi- 
natorial library; to receive a first user input identifying one 
or more regions of interest, each region having a user- 
defined shape and each region c irresponding to alocationin 
each of the scries of images and to a location in the 
combinatorial library; to receive a second user input defining 
a computation for determining a series of values for each of 
the one or more regions of interest, each series of values 
including a value for each of the images; and to determine 
a series of values for each of the one or more regions of 
interest, each value in each series being computed in accor- 
dance with the second user input. 
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Implementations of the invention can include one or more 
of the following advantageous features. The program further 
includes instructions to calculate from each series of values 
for the one or more regions a figure of merit for the library 
member at the corresponding library location. 

In general, in another aspect, the invention features a 
computer program for evaluating a combinatorial chemical 
experiment. The program includes instructions to receive a 
series of images of a combinatorial library, the library 
including a plurality of members, the series of images 
captured from a beginning of the experiment to an end of the 
experiment; to identify, alter the series of images has been 
captured, a plurality of regions of interest, each region 
corresponding to a location in each of the series of images 
and to a location in the combinatorial library; and to deter- 
mine a series of values for one or more regions of interest, 
the series of values for each of the one or more regions 
including a value for each of the images. 

In general, in another aspect, the invention features a 
method l'oi evaluating a combinatorial library including a 
plurality of members. The method includes receiving a 
stream of data including a series of images of the combi- 
natorial library; identifying a plurality of regions of interest, 
each region corresponding to a location i 
of images and to a location in the coi 
determining a series of values for one 
interest, the series of values for each of the one or more 
regions including a value for each of the images; and 
calculating from each series of values for the one or more 
regions a figure of merit for the library member at the 
corresponding library location. 

In general, in another aspect, the invention features a 
computer-implemented method for evaluating a combinato- 
rial chemical experiment. The method includes capturing a 
series of images of a combinatorial library, the library 
including a plurality of members, the series of images being 
captured from a beginning of the experiment to an end of the 
experiment and storing the series of images in memory; after 
the series of images has been captured, identifying a plu- 
rality of regions of interest, each region corresponding 



each of the series 
□material library; 
■ more regions of 



in the combinatorial library; means for receiving a second 
user input defining a computation for determining a series of 
values for each of the one or more regions of interest, each 
series of values comprising a value for each of the images; 
and means for determining a series of values for each of the 
one or more regions of interest, each value in each series 
being computed in accordance with the second user input. 

In general, in other aspect, the invention features a system 
for evaluating a combinatorial chemical experiment. The 

5 system includes means for receiving a series of images of a 
combinatorial library, the library including a plurality of 
members, the series of images captured bom a beginning of 
the experiment to an end of the experiment; means for 
identifying a plurality of regions of interest after the series 

5 of images has been captured, each region corresponding to 
a location in each of the series of images and to a location 
in the combinatorial library; and means for determining a 
series of values for one or more regions of interest, the series 
of values for each of the one or more regions comprising a 

> value for each of the images. 

Advantages that can be seen in implementations of the 
invention include one or more of the following. The use of 
a video image sequence allows the automated extraction of 
data from one or more regions in every frame of the image 

' set. This results in a time-resolved profile of the reaction of 
transformation being observed, rather than merely snapshots 
based on a limited number of selected images or image 
frames. Regions of interest in each image can be defined 
after the image sequence is captured. Calculations and 

' corrections can be applied automatically to every region in 
every frame of the image set. One or more figures of merit 
can be extracted from some or all of the data set for each 
region and can be graphically displayed. 

The details of one or more embodiments of the invention 
are set forth in the accompanying drawings and the descrip- 
tion below. Other features and advantages of the invention 
will become apparent from the description, the drawings, 
and the claims. 

3 BRIEF DESCRIPTION OF THE DRAWINGS 
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FIG. 1 is a block diagram of a data processing system 
showing an implementation of the invention. 

FIG. 2 is a flow diagram of a method of reducing a data 



of the images. 

In general, in one aspect, the invention features a system 
for evaluating a combinatorial library including a plurality 
of members. The system includes means for receiving a 



stream of data comprising a series of images of the combi- , 
nalorial library; means for identifying a plurality of regions 
of interest, each region corresponding to a location in each 
of the series of images and to a location in the combinatorial 
library; means for determining a series of values for one or 
more regions of interest, the series of values for each of the 5 
one or more regions comprising a value for each of the 
images; and means for calculating from each series of values 
for the one or more regions a figure of merit for the library 
member at the corresponding library location. 

In general, in other aspect, the invention features a system t 
for evaluating a combinatorial library, the library including 
a plurality of members. The system includes means for 
receiving a stream of data comprising a series of images of 
the combinatorial library; means for receiving a first user 
input identifying one or more regi insof interest, each region t 
having a user-defined shape and each region corresponding 
to a location in each of the series of images and to a location 



I 1(1. 3 is a How diagram illustrating the method of FIG. 

2 in more detail. 

FIG. 4 is a flow diagram of a method of generating a 

FIG. 5A is a diagram of a combinatorial library. 

FIG. 5B is an illustration of a dialog window for creating 

FIG. 5C is an illustration of selecting the corners of a 

FIG. 6 is a flow diagram of a method of analyzing reduced 
data values and displaying calculated results. 

FIG. 7 is a flow diagram of a method of selecting display 
parameters and a display format. 

FIG. 8A-8C are illustration of formats for displaying 
experimental results. 

FIG. 9 is a window displaying images in data stream. 
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FIG. 10 is a schematic diagram of a thermoelectric device. 

FIG. 11 is a schematic diagram of a thermoelectric device 
including a combinatorial array of thermoelectric materials. 

FIGS. 12A-12B are illustrations of display formats for a 
combinatorial thermoelectric materials experiment. 

FIG. 13 is a schematic diagram of a combinatorial experi- 
ment to monitor heats of reaction. 

lit!. 14 is a diagram illustrating the combinatorial library 
of FIG. 13 in more detail. 10 

FIG. 15 is a diagram of a combinatorial experiment to 
determine phase transition points of a library of materials. 

Fit!. 16 is a schematic diagram of a computer platform 
suitable for implementing the data processing system of the 

Like reference numbers and designations in the various 
drawings indicate like elements. 

DETAILED DESCRIPTION 

mcnt. Tile resulting image data is analyzed by a data 
processing system implementing methods of the present 
invention, as will be described. 

Referring to FIG. 1, camera 100 observes experiment 110 
and records one or more images representing experiment 
110. In the exemplary system that will be described, camera 
100 is a commercially available high performance infrared 
camera, such as the SE-IR CamIRa™, available from SE-IR 
Corporation of Goleta, Calif. Such cameras can be used lo 30 
measure thermal properties such as radiant or photon 
omittance, cxistancc, thermal flux and the like. Commer- 
cially available software, such as SE-IR Corporation's Cam- 
IRa™ software package, can be used to drive the camera's 
operations and to generate the images. Experiment 110 can 35 
be any experiment suitable for monitoring with camera 100. 
It can involve a single reaction vessel containing reagents or 
a single material deposited till a substrate within the field of 
view of camera 100. Alternatively, it can include a combi- 
natorial array or "library" of distinct "members" (separate 40 
vessels of deposited materials), as will be discussed in more 
detail below. 

Image data 120 generated by camera 100 is stored in data 
store 130 as an array of picture elements or ''pixels," each of 
which is represented by a specific intensity or pixel value. 4^ 
Data processing system 140 retrieves image data 120 from 
data store 130; alternatively, data processing system 140 can 
receive image data 120 directly from camera 100 without 
intervening storage in data store 130. After retrieving the 
image data, data processing system 140 processes the image ,,,, 
data in data reduction module 150 and data analysis module 
160 either automatically or under user control. A user can 
interact with system 140 through user interface module 170. 
Data processing system 14(1 displays results through display 
or printing devices 190, 196. 55 

As shown in FIG. 2, data reduction module 170. Data 
pn icessing system 140 display results comprising a series of 
images from data store 130 or camera 100 (step 200). After 
getting the first image in the stream (step 210), data reduc- 
tion module 150 reduces ] he data from that image, as will be 60 
described in further detail below (step 220). If additional 
images remain to be reduced (step. 230), data reduction 
module 150 gets the next image (step 240) and reduces that 
image data (step 220). Data reduction module 150 provides 
the reduced data values to data analysis module 160 (step 65 
250). Optionally, data processing system 140 stores the 
reduced data in data store 130. 
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FIG. 3 describes data reduction step 220 in more detail. 
After receiving an image (step 210), data reduction module 
150 obtains a user-defined mask identifying one or more 
regions of interest in the image (step 300). These regions 
may correspond, for example, to reaction vessel or materials 
that make up the elements of a combinatorial library used in 
experiment 110. Data reduction module 150 applies the 
mask to the image (step 310), and extract a value or values 
for each region of interest (step 320). Data reduction module 
150 uses these values to calculate a reduced data value for 
each region of interest in the image (step 330). This reduced 
data value can be an average of intensity values recorded by 
camera 100 for points within a region of interest. It can also 
be an average change in intensity calculated by subtracting 
a reference point value, for example, a value measured for 
the region at an earlier time (e.g., in a previous image), a 
value measured for background noise or some other value 
(such as reflectance or base line). In other embodiments, the 
reduced dala value can be calculated using other known 
statistical functions. Optionally, dala reduction module 150 
allows the user to select a desired function, for example by 
choosing from a menu of possible functions. Data reduction 
module 150 produces a list of reduced data values for the 
array of regions of interest, which is sent to data analysis 
module 160 (step 340). The list of reduced data values may 
also be stored for future use. Data reduction module 150 
then determines whether additional images remain to be 
reduced (step 230). 

The process of creating a mask is described in FIG. 4. 
I'hrougli user interface 170. the user identifies one 01 more 
regions of interest within an image (step 400). As discussed 
above, these regions will generally correspond to specific 
reaction vessels or materials that make up the elements of a 
combinatorial library monitored by camera 100. Optionally, 
the user can also specify one or more reference regions to 
which experimental data can be compared (step 410). Data 
processing system 140 saves the resulting mask in data store 
130 foi subsequent use during experiment 110, as well as for 
use Liming later experiments sharing a similar substrate 
geometry (step 420). 

FIGS. 5A-5C detail the creation of a mask in one embodi- 
ment for an experiment involving a combinatorial library 
500 of members 510 (FIG. 5A). User interface 170 displays 
"Create Mask" dialog window 520 (FIG. 5B). The user can 
select a grid type 530, as well as the number of horizontal 
and vertical members in the grid, 540 and 550, respectively. 
The user can also select a member type 560, according to the 
nature of experiment 110. In another embodiment, data 
reducing module 150 allows the user to define the shape of 
mask regions to correspond to any shape of library members 
510, for example, through a user interlace similar to those 
found in commercially available drawing and painting soft- 
ware known to those skilled in the art. 

While FIG. 5B shows a "Create Mask" dialog for ther- 
moelectric materials, the nature of the experiment is not a 
limiting feature for this invention as discussed below. The 
user can select a particular member length or width 570. 
After selecting the "Create" button, the user graphically 
specifies three corners 580 of the grid, clicks on the image 
and connecting lines are draw n (FIG. 5C). The elements of 
the grid can be moved as groups or individually to accom- 
modate spacial irregularities in the physical substrate. Data 
reduction module 150 uses this information to create the 

As shown in FIG. 6, data analysis module 160 gets the list 
of reduced data values for each image from data reduction 
module 150 or data store 130 (step 600). Data analysis 
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module 160 uses the values to calculate experimental 
results, including one or more figures of merit for each 



region of interest (step 610), as will be described 
detail below. The user can save the calculated results (steps 
620 and 630), andean view iln displayed results in a number 
of graphical formats (steps 620 and 640). 

The user's interaction with data analysis module 160 
(steps 620 through 640) is described in more detail in 
reference to FIG. 7. Through user interf ace f 70, the user can 
select a variety of display options (step 700). These include 
a number of display formats (step 710) illustrated in FIGS. 
8A-8C, such as an array of ••thumbnail" graphs 800 plotting 
experimental data (for example, intensity versus time) for 
each region of interest a graph of experimental data for any 
selected region of interest (a "zoom-in" graph) 850, or a 
three-dimensional histogram 880 displaying a calculated 
value for each region of interest. Data analysis module 
retrieves display partners and control information (step 720) 
and gets the results calculated at step 610 (step 730). Data 
analysis module 160 displays the results in the selected 
display format (step 740). User selection step 7(K) also 
includes the option to choose display parameters (step 750). 
These include parameters such as histogram scale, range and 
colors, or time and value ranges for thumbnail or zoom-in 
graphs. By setting these parameters, the user can view 
experimental results over a specified time period, with the 
value at any specified time subtracted to show changes in 
temperature, or with a reference value (as described above) 
subtracted to remove background (or systematic) changes. 

As shown in FIG. 9, the 
image or images recorded by camera 100. Window 
displays an infrared image 910 of experiment 110, such as 
microtiter plate 920. Depending on the user-selected display 
parameters, different colors in infrared image 910 depict the 
range of heat emitted by objects within the field of camera 
100. The user can navigate through the steam of images 
using buttons 930«-e provided in pane 940. Buttons 93<W 
and 9306 navigate forward or backward, respectively, by a 
single frame. Buttons 930c and 930./ pr. .vide "fast forward" 
and "rewind" functions, respectively. In one embodiment, in 
"fast forward" mode data analysis module 160 and/or user 
interface module 170 determines the number of frames 
remaining in the stream of images and displays selected 
frames until it reaches the end of the stream. Similarly, ii 
"rewind" mode, the nuinbei or preceding frames 
mined and selected frames are displayed in revei 
until the beginning of the image stream is reached. I'hus, for 
example, fast forward mode can display every tenth frame in 
a stream including fOOO frames, for a total of fOO frames 
displayed. Pane 940 also displays the frame number of the 
image displayed. 

Data processing system 140 can be used to analyze 
material or chemical systems thai lend themselves to char- 
n by IR thermography. Using IR thermography, 
;xistance) of an entire library of materials 
can be monitored wdth an infrared camera and the measured 
values can be used to derive a variety of thermodynamic 
properties associated with the materials. Commercial posi- 
tion sensitive systems such as infrared local plane arrays can 
have a high sensitivity over a wide temperature range. 
Commercial infrared cameras incorporating such systems 
can acquire data at speeds up to fOO or more frames per 
second. In combination with such a system, data processing 
system 140 is particularly useful as a tool for quickly 
screening large numbers of materials for desirable thermo- 
dynamic characteristics. In one embodiment, data process- 
ing system 140 receives images at a rate of 12 to 20 frames 



per second and reduces the data by averaging values for each 
frame to obtain the equivalent of one averaged frame of data 
per second, thereby increasing the signal to noise ratio of the 
experimental results. 

Depending on the process being monitored, different data 
acquisition rates — for example, rates of about f ,6, f 2 or 20 
frames per second — provide a data stream suitable for 
deriving a reaction profile sufficient to track many chemical 
material transformations and thermal diffusion 
Higher data acquisition rates are useful in char- 
acterizing processes requiring a relatively high degree of 
resolution. For example, metal compositions can be charac- 
terized based on the range of temperatures at which melting 
occurs. For many metal compositions, melting begins at one 
temperature and continues until the material reaches a higher 
temperature. The first temperature, below which the entire 
sample is in a solid phase, is known as the solidus tempera- 
ture; the second, above which the entire sample is in a liquid 
phase, is known as the liquidus temperature. To identify 
eutectic compositions — those in which the solidus tempera- 
ture and liquidus temperature are equal — it is useful to 
screen an array of metals covering a range of compositions 
for those having the lowest liquidus temperature. By rapidly 
heating such an array, it is possible to exceed a composi- 
tion's solidus temperature without the onset of melting due 
to slow diffusion in the solid stale. II heating is sufficiently 
rapid — for example at 20° to 60° C. per second — it is 
he original 30 possible to reach the liquidus temperature without melting, 
at which point the entire sample quickly melts. Data acqui- 
sition at speeds of 18 to 20 frames per second are sufficient 
to resolve such rapid thermody namic changes. 

The features and advantages of the invention can be 
appreciated from a description of its application to a par- 
ticular implementation in the field of infrared 
thermography — the characterization of thermoelectric male- 
Thermoelectric Materials 

Thermoelectric devices are solid state devices that pump 
heat from one junction to another when subjected to an 
deter- 45 electric current, a phenomenon known as the Peltier effect, 
order Thermoelectric materials are characterized by a number of 
physical parameters including the thermal conductivity k, 
the electrical conductivity a, the Seebeck coefficient S, the 
Hall coefficient R H , the charge carrier mobility fi, the device 
operating temperature T, the change carrier effective mass 
m*, and the band gap E^. An estimate of the thermoelectric 
device elliciency lor pumping heal relative to the heat lost 
due to the electrical resistivity is the thermoelectric figure of 
merit defined by: 

S 2 <tT 



) FIG. 10 illustrates a model thermoelectric device con- 
structed from a thermoelectric material 1000 connected to 
voltage source 1011) with voltage wires 1020 made of a high 
conductivity metal. Under steady state or adiabatic 
conditions, the heat pumped by the Peltier effect will be 

5 equal to the heat carried by the thermal conduction, which 
leads to the following relationship for the thermoelectric 
figure of merit: 
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Op,. and Vj,„„,. art the amount of heal transported by the 
Peltier effect and the amount of energy lost to Joule heating, 
respectively; P Pehier and P Joule are each the corresponding 
power (heat per unit time). The expression for the total 
power dissipated in the device is given by the sum of the 
thermoelectric (Peltier) and Joule components; 



where the first term is the powder dissipated by Joule effects 
(P/ox/e) and the second term is the contribution from the 
Peltier component (P (v/ „,. r ); K is the electrical resistance of 
the circuit and II is the Peltier coefficient of the thermoelec- 
tric material measured relative to the material that makes up i<> 
the voltage contacts. Application of an oscillatory voltage at 
a reference frequency co 0 , such as V(t)=V 0 cos(oj 0 f), causes 
the Joule term to oscillate at twice the reference frequency 
(2oj, ,) due to the voltage being squared, while the Peltier 
term is linear in voltage and is observed at the reference 25 
frequency co 0 . Measurement of the power dissipated at the 
junction between the voltage contact and the thermoelectric 
material as a function of time, P(t), followed by a Fourier 
transform to power as a function of frequency, P(cu), allows 
the contribution due to. Ionic effects lo he distinguished from 30 
(and compared to) the contribution due lo Peltier effects. The 
ratio of the two amplitudes P(<o p ) P(2(.> () ) is the thermoelec- 
tric figure of merit ZT. 

The present invention provides a fast and efficient screen 
for the identification of thermoelectric materials in a com- 35 
binatorial library of candidate materials, as well as an 
analytical tool for characterizing such materials by deter- 
mining the thermoelectric figure of merit. Suitable libraries 
can be created using techniques such as those described in 
U.S. Pat. No. 5,776,359, which is incorporated herein by 40 
reference in its entirety. 

As shown in FIG. 11, in combinatorial thermoelectric 
device 1100, voltage source 1110 applies a parallel voltage 
through voltage wires 1120 to all members 1130 in a 
combinatorial library 1140. Library 1140 is made from a 45 
material with low thermal conductance, such as 25 lo 50 
micron thick polyimide sheets to minimize heat lost to the 
substrate holding elements 1130: for the same reason, library 
1140 is maintained in an evacuated environment lo minimize 
heat lost to the surroundings. To maintain adiabatic 50 
conditions, the voltage applied by voltage source 1110 is 
kept as small as possible, with reference frequencies typi- 
cally being on the order of 0.1 Hz or less. Voltage wires 1120 
are made of a high conductivity metal such as Ag, Au, Cr, 
Ta, or Cu to maintain good electrical contact and because 55 
their Peltier coefficients are low relative to those found in 
semiconducting materials. 

As discussed more generally above, data reduction mod- 
ule 150 obtains a series of images of thermoelectric device 
1100 from infrared camera 100. Data reduction module 150 60 
generates reduced dala values as described above, and 
provides those values to data analysis module 160. 

With these reduced data values, data analysis module 160 
calculates the intensity as a function of frequency for each 
region of interest using fast Fourier transform. As shown in 65 
FIGS. 12A and 12R, the user can view the data 
geographically, in either the time domain 1200 or the 



frequency domain 1250. Data analysis module 160 uses the 
frequency data to calculate the thermoelectric figure of merit 
for each region of interest as described above. The figure of 
merit (1220, 1270) is displayed in thumbnail graphs 1210 
and 1260. The user can also view a three dimensional 
histogram displaying the figure of merit for each region of 
interest, as described above. 

Chemical Transformations 

J Data processing system 140 can also be used to monitor 
the progress of chemical reactions or transformations. In 
many embodiments, the reaction is a catalytic reaction and 
catalyst activity is being determined, which is the example 
described below. Referring to FIG. 13, in this application an 
' infrared camera 100 monitors emittance associated with heat 
evolution or absorption or with a compound or chemical's 
disappearance from or appearance in a combinatorial library 
1310 under various external conditions such as temperature 
and gas How. For example, if a solid catalyst library and its 
surrounding support in a two-dimensional library are 
exposed to a reactant, a measurable heating of the surround- 
ings may occur depending on the activity of the chemical 
process. Such libraries can be generated using techniques 
such as those disclosed in U.S. application Scr.^No. 08/898, 
' 715, filed Jul. 22, 1997, or Ser. No. 09/227,558, filed Jan. 8, 
1000, which are both incorporated herein by reference in 
their entirety. In the case of a catalyst, the activity of the 
catalyst correlates to the energy released or absorbed as heat 
during the chemical reaction between the catalyst and the 
exchange gas. In a combinatorial library 1310, members 
1320 are nearly identical in thermal mass, so measurements 
of the heat evolved by one element in the library relative to 
others within the library can be used to characterize the 
chemical processes induced by these materials. 

FIG. 14 illustrates a two-dimensional library 1400 of 
materials useful in this embodiment. The individual library 
members are reaction wells 1410 in a substrate 1420. 
Substrate 1420 is placed within a sealed reaction chamber 
which is pressurized with the relevant gas. Windows 1430 
and 1440 are made of an infrared transparent medium (such 
as sapphire, antireflection coated silicon, BaF 2 , CaF 2 or 
NaCl) capable of maintaining the pressurized gas inside the 
chamber. Since window 1430 and 1440 are transparent to 
infrared radiation, thermal imagine, techniques can be used 
to monitor the heat of reaction of each library clement under 
various external conditions. 

Measuring the heal generated by a catalytic transforma- 
tion is a powerful technique for rapidly screening catalyst 
efficacy. For condensed phase reactions, there is direct 
thermal contact between the catalyst, products, and solvent. 
I tins, the library can he directh imaged and the cmissivity 
of the samples will be roughly that of the pure solvent, 
allow ing direct comparison of the members in a library. In 
cases where cmissivity differences exist between materials 
in the library, direct comparison of library members is more 
complicated, but the experimental results are still useful in 
screening for raw catalyst activity. Differences in emissivity 
may also be dealt with by imaging the library through a 
material (such as graphite). This technique is particularly 
useful for screening gas phase reactions, where the catalysts 
is mounted on a support and large emissivity differences are 
common. In this case, the reaction can be monitored from 
the backside of the catalyst support. 

In a typical experiment, a library is loaded into a sample 
chamber with an IR transparent window and the entire 
system is allowed to come to thermal equilibrium. The 
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sample is monitored with camera 100 to establish a back- 
ground reading, and the chamber is pressurized with a 
reactant gas. The sample is monitored by camera 100 over 
the course of the catalytic reaction, for example, for about 
one hour. The resultant data is analyzed as will be described : 
below. The change in emittance is directly proportional to 
the catalytic activity of the library member. 

As discussed more generally above, data reduction mod- 
ule 150 obtains a series of images of library 1400 from 
infrared camera 100. Data reduction module 150 generates ] 
reduced data values as described above, and furnishes those 
values to data analysis module 160. 

With these reduced data values, data analysis module 160 
calculates the integral of intensity (proportional to the total 
af starting material). As described above, the 1 
k the data in the form of an array of graphs 800 
.'ersus time and integral of intensity versus time. 
Each thumbnail graph also depicts two figures of merit for 
each region of interest — the total integrated intensity and the 
maximum intensity for the element. The user can also view 2 
a three dimensional histogram displaying cither figure of 
merit for each region of interest. 

Metals 

Data processing system van also be used lo eharavleri/e 
melting points of metals or other materials (e.g., composite 
materials) using infrared imaging. In this application, an 
infrared camera 100 is used to monitor a library whose 
members include a series of metals or other materials. 3 
Suitable libraries can be generated using icclinK]ucs such as 
those described in U.S. Pat. No. 5,776,359, which is incor- 
porated herein by reference in its entirety. Camera 100 
measures intensities that are a function of temperature and 
emissivity of each member. Changes in the structure and , 
bonding of a chemical composition during a transition from 
one thermodynamic phase to another result in a change in 
the composition's emissivity. During a phase transition the 
temperature of a library member may change or the rate of 
temperature change may increase or decrease. The thermo- 
dynamics of given material's phase transition can be char- 
acterized by comparing I he emissivity changes over a given 
temperature range with that of a standard material having a 
constant emissivity in the relevant temperature range. 

The infrared camera 100 monitors the intensity from 4 
every library member in parallel and compares it to the 
intensity of a known standard material within the field of 
view of the camera (which may be the substrate holding the 
library) and subjected to lite same physical conditions as the 
library members. In this way, complicated phase relation- 5 
ships arc measured for large libraries of materials by heating 
or cooling the library and deriving changes in the emissivity. 

Referring to FIG. 15, as discussed more generally above, 



n this embodin 



a 100 rt 



library 1500, including library members 1510, as the tem- 
perature is varied over a range of interest. Data reduction 
module 150 obtains a series of images of library 1500 and 
generates reduced data values as described above. Data 
reduction module 150 furnishes those values to data analysis 
module 160. 

Using these reduced data values, data analysis module 
160 calculates the relative emissivity of each library member 
as compared to the standard. After smoothing the data, data 
analysis module 160 calculates the derivative of the ratio of 
library member emissivity to standard. The maximum rate of • 
change occurs a the material's melting point which here is 
the figure of merit. As described above, the user can view the 



experimental results as an array of thumbnail graphs 800 
depicting a plot of intensity versus time for a given library 
member, intensity versus time for the standard sample 
(which may be the substrate) and a plot that is a ratio of the 
first two plots. The user can also view a reduced plot of 
melting point as a [unction of library member. The figure of 
merit — the melting point calculated as described above — is 
also displayed. As described above, the user can also veiw 
the data as a three dimensional histogram displaying either 
figure of merit for each library element. 

The invention can be implemented in digital electronic 
circuitry, or in computer hardware, firmware, software, or in 
combinations of them. Apparatus of the invention can be 
implemented in a computer program product tangibly 
embodied in a machine-readable storage device for execu- 
tion by a programmable processor; and method steps of the 
invention can be performed by a programmable processor 
executing a program of instructions to perform functions of 
the invention by operating on input data and generating 
output. The invention can be implemented advantageously 
in one or more computer programs that arc executable on a 
programmable system including at least one programmable 
processor coupled to receive data and instructions from, and 
to transmit data and instructions to, a data storage system, at 
least one input device, and at least one output device. Each 
computer program can be implemented in a high-level 
procedural or object-oriented programming language, or in 
assemble or machine language if desired: and in any case, 
the language can be a compiled or interpreted language. 
Suitable processors include, by way of example, both gen- 
eral and special purpose microprocessors. Generally, a pro- 
cessor will receive instructions and data from a read-only 
memory and/or a random access memory. Storage devices 
suitable for tangibly embodying computer program instruc- 
tions and data include all forms of non-volatile memory, 
including bv wav of example semiconductor memory 
devices, such as EPROM, EEPROM, and flash memory 
devices; magnetic disks such as internal hard disks and 
removable disks; magneto-optical disks; and CD-ROM 
disks. Data can also be temporarily stored in volatile 
memory. Any of the foregoing can be supplemented by, oi 
incorporated in, ASICs (application-specific integrated 

To provide for interaction with a user, the invention can 
be implemented on a computer system having a display 
device such as a monitor or LCD screen for displaying 
information to the user and a keyboard and a pointing device 
such as a mouse or a trackball by which the user can provide 
input to the computer system. The computer system can be 
programmed to provide a graphical user interface through 
which computer programs interact with users. 

An example of one such type of computer is shown in 
FIG. 16, which shows a block diagram of a programmable 
processing system (system) 1600 suitable for implementing 
or performing the apparatus or methods of the invention. 
The system 1600 includes a processor 1610, a random 
access memory ( k \M) 1620, a program memory 1630 (for 
example, a writable read-only memory (ROM) such as a 
flash (ROM), a hard drive controller 1640, and an input/ 
output (I O) controller 1650 coupled by a processor (CPU) 
bus 1660. The system 1600 can be preprogrammed, in 
ROM, for example, or it can be programmed (and 
reprogrammed) by loading a program from another source 
(for example, from a floppy disk, a CD-ROM, or another 
computer). 

The hard drive controller 1640 is coupled to a hard disk 
1645 suitable for storing executable computer programs, 
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including programs embodying the present invention, and 
data including the images, masks, reduced data values and 
calculated results used in and generated by the invention. 

The I/O controller 1650 is coupled by means of an I/O bus 
1670 to an I/O interface 1680. The I/O interface 1680 : 
receives and transmits data (e.g., stills, pictures, movies, and 
animations for importing into a composition) in analog or 
digital 1'unii over communication links such as a serial link, 
local area network, wireless link, and parallel link. 

Also coupled to the I/O bus 1670 is a display 1690 and a 3 
keyboard 1695. Alternatively, separate connections 
(separate buses) can be used for the I/O interface 1670, 
display 1690 and keyboard 1695. 

The invention has been described in terms of particular 
embodiments. Other embodiments are within the scope of } 
the following claims. For example, the invention has been 
described as being implemented in a number of different 
embodiments, each intended j<> monitor, analyze and display 
data for a different experiment. Alternatively, a number of 
different applications can he implemented in a single system, 
for example, where data processing system 140 includes 
multiple versions of data analysis module 140, each 
designed to analyze and display data from a different type of 
experiment. Also, data structures other than the ones men- 
tioned above can be used in storing and processing data, for , 
example, mask and image information can be encapsulated 
in objects and stored in an object oriented database. In 
addition, the steps of the invention can be performed in a 
different order and still achieve desirable results. 

What is claimed is: , 

1 . A computer program on a computer-readable medium 
for evaluating a corobinatorial library, the program compris- 



receive a stream of data comprising a series of images of 
a combinatorial library, the combinatorial library - 
including a plurality of members, each member being 
assoeialed with a location in the combinatorial library; 

receive user input identifying a plurality of regions of 
interest, each region corresponding to a location in each 
of the series of images and to a location in the combi- , 
nalorial library: 

determine a series of values for one or more regions of 
interest, the series of values for each of the one or more 
regions comprising a value for each of the images; and 

calculate from each series of values for the one or more < 
regions a figure of merit for the library member at the 
corresponding library location. 

2. The computer program of claim 1, wherein each region 
of interest corresp' nids to a plurality of pixels in each image 
and the series of values for a region of interest comprises an ; 
average value for each of the corresponding plurality of 
pixels in each of the images. 

3. The computer program of claim 2, wherein the stream 
of data comprises a series of images genera ted at a frequency 
of greater than about 1 frame per second. 

4. The computer program of claim 2, wherein the stream 
of data comprises a series of images generated at a frequency 
of greater than about 6 frames per second. 

5. The computer program of claim 2, wherein the stream 
of data comprises a scries of images generated at a frequency < 
of greater than about 12 frames per second. 

6. The computer program of claim 2, wherein the stream 
of data comprises a series of images generated at a frequency 
of greater than about 20 frames per second. 

7. The computer program of claim 2, further comprising < 
instructions to display a graphical representation of the 
figures of merit. 



8. The computer program of claim 7, wherein the graphi- 
cal representation comprises a histogram. 

9. The computer program of claim 1, wherein the series of 
images is captured from a first time during a combinatorial 
experiment to a second time during the combinatorial 
experiment. 

10. The computer program of claim 1, wherein the series 
of images comprises a series of infrared images. 

11. The computer program of claim 10, wherein the figure 
of merit comprises an emittance change for a librarv mem- 
ber. 

12. The computer program of claim 10, wherein the figure 
of merit comprises a phase transition point for a library 
member. 

13. The computer program of claim 10, wherein the figure 
of merit comprises a thermoelectric figure of merit for a 
library member. 

14. The computer program of claim 1, wherein the 
instructions to identify a plurality of regions of interest 
comprise instructions to receive a first user input identifying 
one or more regions of interest, each region having a 
user-defined shape, the computer program further compris- 
ing instructions to receive a second user input defining a 
computation for determining a series of values for each of 
the one or more regions of interest. 

15. The computer program of claim 14, wherein the 
instructions to determine a series of values for each of the 



n accordance with (he 



compute each valve ir 
second user input. 

16. The computer program of claim 15, wherein the 
computation is an average for a group of pixels correspond- 
ing to a region of interest. 

17. The computer program of claim 1, wherein the plu- 
rality of regions of interest is identified after the series of 
images has been captured. 

18. Acomputcr program on a computer-readable medium 
for evaluating a corobinatorial library, the program compris- 
ing instructions to: 

receive a stream of data comprising a series of images of 
a combinatorial library, the combinatorial library 
including a plurality of members, each member being 
associated with a location in the combinatorial library; 

receive a first user input identifying one or more regions 
of interest, each region having a user-defined shape and 
each region corresponding to a location in each of the 
series of images and to a location in the combinatorial 
library; 

receive a second user input defining a computation for 
determining a series of values for each of the one or 
more regions of interest, each series of values com- 
prising a value for each of the images; and 

determine a series of values for each of the one or more 
regions of interest, each value in each series being 
computed in accordance with the second user input. 

19. The computer program of claim 18, further compris- 
ing instructions to calculate from each series of values for 
the one or more regions a figure of merit for the library 
member at the corresponding library location. 

20. A computer program for evaluating a combinatorial 
chemical experiment, the program comprising ir 



receives a series of images of a combinatorial library, the 
library including a plurality of members, each member 
being associated with a location in the combinatorial 
library, the series of images being captured from a first 
time during the experiment to a second time during the 
experiment; 
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after the series of images has been received, receive user 
input identifying a plurality of regions of interest, each 
region corresponding to a location in each of the scries 
ofimages andto a location in the combinatorial library; 

determine a series of values for one or more regions of 
interest, the series of values for each of the one or more 
regions comprising a value for each of the images. 

21. The computer program of claim 20, further compris- 
ing instructions to calculate from each series of values for 10 
the one or more regions a figure of merit for a library 
member at the corresponding library location. 

22. The computer program of claim 21, further compris- 
ing instructions to display a graphical representation of each 
figure of merit. 15 

23. A method for evaluating a combinatorial library, the 
method comprising: 

receiving a stream of data comprising a series of images 
of a combinatorial library, the combinatorial library 
including a plurality of members, each member being J! 
associated with a location in the combrnatorral library; 

receive user input identifying a plurality of regions of 
interest, each region coresponding to a location in each 
of the series of images and to a location in the combi- 25 
natorial library; 

determining a series of values for one or more regions of 
interest, the series of values for each of the one or more 
regions comprising a value for each of the images; and 

calculating for each series of values for the one or more 30 
regions a figure of merit for the library member at the 
corresponding library location. 

24. The method of claim 23, wherein each region of 
interest corresponds to a plurality of pixels in each image 
and the series of values for a region of interest comprises an 35 
average value for each of the corresponding plurality of 
pixels in each of the images. 

25. The method of claim 24, wherein the stream of data 
comprises a series of images generated al a frequency of 
greater than about 1 frame per second. 40 

26. The method of claim 24, wherein the stream of data 
comprises a series of images generated at a frequency of 
greater than about 6 frames per second. 

27. The method of claim 24, wherein the stream of data 
comprises a series of images generated at a frequency of 45 
greater than about 12 frames per second. 

28. The method of claim 24, wherein the stream of data 
comprises a series of images generated at a frequency of 
greater than about 20 frames per second. 

29. The method of claim 24, further comprising display- 50 
ing a graphical representation of the figures of merit. 

30. The method of claim 29, wherein the graphical 
representation comprises a histogram 



31. The method of claim 23, wherein the series ofimages 
is captured from a first time during a combinatorial experi- 
ment to a second time during the combinatorial experiment. 

32. The method of claim 31, wherein the series of images 
comprises a series of infrared images. 

33. The method of claim 32, wherein the figure of merit 
comprises an emittance change for a library member. 

34. The method of claim 32, wherein the figure of merit 
comprises a phase transition point for a library member. 

35. The method of claim 32, wherein the figure of merit 
comprises a thermoelectric figure of merit for a library 
member. 

36. The method of claim 23, wherein identifying a plu- 
rality of regions of interest comprises receiving a first user 
input identifying one or more regions of interest, each region 
having a user-defined shape, the method further comprising 
receiving a second user input defining a computation for 
determining a series of values for each of the one or more 
regions of interest. 

37. The method of claim 36, wherein determining a series 
of values for each of the one or more regions of interest 
comprises computing each value in each series in accor- 
dance with the second user input. 

38. The method of claim 37, wherein the computation is 
an average for a group of pixels corresponding to a region 
of interest 

39. lire method of claim 23. wherein the plurality of 
regions of interest is identified after the series of images has 
been captured. 

40. A computer-implemented method for evaluating a 
combinatorial chemical experiment, comprising: 

receiving a series of images of a combinatorial library, the 
library including a plurality of members, the combina- 
torral library including a plurality of members, each 
member being associated with a location in the com- 
binatorial library, the series of images being captured 
from a first time during the experiment to a second time 
during the experiment and storing the series of images 
in memory; 

after the series of images has been received, receiving 
riser input identifying a plurality ol regions of interest, 
each region corresponding to a location in each of the 
scries of images and to a location in lire coiiibinalorial 

library; and 

determining a scries of values for one or more regions of 
interest, the seiies of values for each of the one or more 
regions comprising a value for each of the images. 

41. The method of claim 40, further comprising calculat- 
ing from each series of values for the one or more regions 
a figure of merit for a library member at the corresponding 
library location. 

42. The method of claim 41, further comprising display- 
ing a graphical representation of each figure of merit. 
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